Visual features are processed before navigational affordances in the human brain.

To navigate through their immediate environment humans process scene information rapidly. How does the cascade of neural processing elicited by scene viewing to facilitate navigational planning unfold over time? To investigate, we recorded human brain responses to visual scenes with electroencephalography and related those to computational models that operationalize three aspects of scene processing (2D, 3D, and semantic information), as well as to a behavioral model capturing navigational affordances. We found a temporal processing hierarchy: navigational affordance is processed later than the other scene features (2D, 3D, and semantic) investigated. This reveals the temporal order with which the human brain computes complex scene information and suggests that the brain leverages these pieces of information to plan navigation.

Revisiting the self-generation effect in proofreading.

The self-generation effect refers to the finding that people's memory for information tends to be better when they generate it themselves. Counterintuitively, when proofreading, this effect may make it more difficult to detect mistakes in one's own writing than in others' writing. We investigated the self-generation effect and sources of individual differences in proofreading performance in two eye-tracking experiments. Experiment 1 failed to reveal a self-generation effect. Experiment 2 used a studying manipulation to induce overfamiliarity for self-generated text, revealing a weak but non-significant self-generation effect. Overall, word errors (i.e., wrong words) were detected less often than non-word errors (i.e., misspellings), and function word errors were detected less often than content word errors. Fluid intelligence predicted proofreading performance, whereas reading comprehension, working memory capacity, processing speed, and indicators of miserly cognitive processing did not. Students who made more text fixations and spent more time proofreading detected more errors.

Adaptive visual selection in feature space.

Visual perception relies on efficient selection of task-relevant information for prioritized processing. A prevalent mode of selection is feature-based selection, and a key question in the literature is the shape of the selection profile-that is, when a feature is selected, what is the landscape of priority for all features in that dimension? Past studies have reported conflicting findings with both monotonic and nonmonotonic profiles. We hypothesized that feature selection can be adaptively adjusted based on stimulus factors (feature competition) and task demands (selection precision). In three experiments, we manipulated these contextual factors in a central task while measuring selection profile in a peripheral task. We found a nonmonotonic, surround suppression, profile when feature competition and selection precision was high, but observed a monotonic profile when these factors were low. Furthermore, manipulation of selection precision alone can shape selection profile independent of feature competition. These findings reconcile previous conflicting results and importantly, demonstrate that feature selection is highly adaptive, allowing flexible allocation of processing resources to ensure efficient extraction of visual information.

CX-ToM: Counterfactual explanations with theory-of-mind for enhancing human trust in image recognition models.

We propose CX-ToM, short for counterfactual explanations with theory-of-mind, a new explainable AI (XAI) framework for explaining decisions made by a deep convolutional neural network (CNN). In contrast to the current methods in XAI that generate explanations as a single shot response, we pose explanation as an iterative communication process, i.e., dialogue between the machine and human user. More concretely, our CX-ToM framework generates a sequence of explanations in a dialogue by mediating the differences between the minds of the machine and human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly modeling the human's intention, the machine's mind as inferred by the human, as well as human's mind as inferred by the machine. Moreover, most state-of-the-art XAI frameworks provide attention (or heat map) based explanations. In our work, we show that these attention-based explanations are not sufficient for increasing human trust in the underlying CNN model. In CX-ToM, we instead use counterfactual explanations called fault-lines which we define as follows: given an input image I for which a CNN classification model M predicts class c pred , a fault-line identifies the minimal semantic-level features (e.g., stripes on zebra), referred to as explainable concepts, that need to be added to or deleted from I to alter the classification category of I by M to another specified class c alt . Extensive experiments verify our hypotheses, demonstrating that our CX-ToM significantly outperforms the state-of-the-art XAI models.

Unsupervised EEG Artifact Detection and Correction.

Electroencephalography (EEG) is used in the diagnosis, monitoring, and prognostication of many neurological ailments including seizure, coma, sleep disorders, brain injury, and behavioral abnormalities. One of the primary challenges of EEG data is its sensitivity to a breadth of non-stationary noises caused by physiological-, movement-, and equipment-related artifacts. Existing solutions to artifact detection are deficient because they require experts to manually explore and annotate data for artifact segments. Existing solutions to artifact correction or removal are deficient because they assume that the incidence and specific characteristics of artifacts are similar across both subjects and tasks (i.e., "one-size-fits-all"). In this paper, we describe a novel EEG noise-reduction method that uses representation learning to perform patient- and task-specific artifact detection and correction. More specifically, our method extracts 58 clinically relevant features and applies an ensemble of unsupervised outlier detection algorithms to identify EEG artifacts that are unique to a given task and subject. The artifact segments are then passed to a deep encoder-decoder network for unsupervised artifact correction. We compared the performance of classification models trained with and without our method and observed a 10% relative improvement in performance when using our approach. Our method provides a flexible end-to-end unsupervised framework that can be applied to novel EEG data without the need for expert supervision and can be used for a variety of clinical decision tasks, including coma prognostication and degenerative illness detection. By making our method, code, and data publicly available, our work provides a tool that is of both immediate practical utility and may also serve as an important foundation for future efforts in this domain.